Neural signatures of shared subjective affective engagement and disengagement during movie viewing.

When watching a negative emotional movie, we differ from person to person in the ease with which we engage and the difficulty with which we disengage throughout a temporally evolving narrative. We investigated neural responses of emotional processing, by considering inter-individual synchronization in subjective emotional engagement and disengagement. The neural underpinnings of these shared responses are ideally studied in naturalistic scenarios like movie viewing, wherein individuals emotionally engage and disengage at their own time and pace throughout the course of a narrative. Despite the rich data that naturalistic designs can bring to the study, there is a challenge in determining time-resolved behavioral markers of subjective engagement and disengagement and their underlying neural responses. We used a within-subject cross-over design instructing 22 subjects to watch clips of either neutral or sad content while undergoing functional magnetic resonance imaging (fMRI). Participants watched the same movies a second time while continuously annotating the perceived emotional intensity, thus enabling the mapping of brain activity and emotional experience. Our analyses revealed that between-participant similarity in waxing (engagement) and waning (disengagement) of emotional intensity was directly related to the between-participant similarity in spatiotemporal patterns of brain activation during the movie(s). Similar patterns of engagement reflected common activation in the bilateral ventromedial prefrontal cortex, regions often involved in self-referenced evaluation and generation of negative emotions. Similar patterns of disengagement reflected common activation in central executive and default mode network regions often involved in top-down emotion regulation. Together this work helps to better understand cognitive and neural mechanisms underpinning engagement and disengagement from emotionally evocative narratives.

Collective incentives reduce over-exploitation of social information in unconstrained human groups.

Collective dynamics emerge from countless individual decisions. Yet, we poorly understand the processes governing dynamically-interacting individuals in human collectives under realistic conditions. We present a naturalistic immersive-reality experiment where groups of participants searched for rewards in different environments, studying how individuals weigh personal and social information and how this shapes individual and collective outcomes. Capturing high-resolution visual-spatial data, behavioral analyses revealed individual-level gains-but group-level losses-of high social information use and spatial proximity in environments with concentrated (vs. distributed) resources. Incentivizing participants at the group (vs. individual) level facilitated adaptation to concentrated environments, buffering apparently excessive scrounging. To infer discrete choices from unconstrained interactions and uncover the underlying decision mechanisms, we developed an unsupervised Social Hidden Markov Decision model. Computational results showed that participants were more sensitive to social information in concentrated environments frequently switching to a social relocation state where they approach successful group members. Group-level incentives reduced participants' overall responsiveness to social information and promoted higher selectivity over time. Finally, mapping group-level spatio-temporal dynamics through time-lagged regressions revealed a collective exploration-exploitation trade-off across different timescales. Our study unravels the processes linking individual-level strategies to emerging collective dynamics, and provides tools to investigate decision-making in freely-interacting collectives.

The scaling of mental computation in a sorting task.

Many cognitive models provide valuable insights into human behavior. Yet the algorithmic complexity of candidate models can fail to capture how human reaction times scale with increasing input complexity. In the current work, we investigate the algorithms underlying human cognitive processes. Computer science characterizes algorithms by their time and space complexity scaling with problem size. We propose to use participants' reaction times to study how human computations scale with increasing input complexity. We tested this approach in a task where participants had to sort sequences of rectangles by their size. Our results showed that reaction times scaled close to linearly with sequence length and that participants learned and actively used latent structure whenever it was provided. This behavior was in line with a computational model that used the observed sequences to form hypotheses about the latent structures, searching through candidate hypotheses in a directed fashion. These results enrich our understanding of plausible cognitive models for efficient mental sorting and pave the way for future studies using reaction times to investigate the scaling of mental computations across psychological domains.

Developmental changes in exploration resemble stochastic optimization.

Human development is often described as a 'cooling off' process, analogous to stochastic optimization algorithms that implement a gradual reduction in randomness over time. Yet there is ambiguity in how to interpret this analogy, due to a lack of concrete empirical comparisons. Using data from n = 281 participants ages 5 to 55, we show that cooling off does not only apply to the single dimension of randomness. Rather, human development resembles an optimization process of multiple learning parameters, for example, reward generalization, uncertainty-directed exploration and random temperature. Rapid changes in parameters occur during childhood, but these changes plateau and converge to efficient values in adulthood. We show that while the developmental trajectory of human parameters is strikingly similar to several stochastic optimization algorithms, there are important differences in convergence. None of the optimization algorithms tested were able to discover reliably better regions of the strategy space than adult participants on this task.

Representational exchange in social learning: Blurring the lines between the ritual and instrumental.

We propose that human social learning is subject to a trade-off between the cost of performing a computation and the flexibility of its outputs. Viewing social learning through this lens sheds light on cases that seem to violate bifocal stance theory (BST) - such as high-fidelity imitation in instrumental action - and provides a mechanism by which causal insight can be bootstrapped from imitation of cultural practices.

Time pressure changes how people explore and respond to uncertainty.

How does time pressure influence exploration and decision-making? We investigated this question with several four-armed bandit tasks manipulating (within subjects) expected reward, uncertainty, and time pressure (limited vs. unlimited). With limited time, people have less opportunity to perform costly computations, thus shifting the cost-benefit balance of different exploration strategies. Through behavioral, reinforcement learning (RL), reaction time (RT), and evidence accumulation analyses, we show that time pressure changes how people explore and respond to uncertainty. Specifically, participants reduced their uncertainty-directed exploration under time pressure, were less value-directed, and repeated choices more often. Since our analyses relate uncertainty to slower responses and dampened evidence accumulation (i.e., drift rates), this demonstrates a resource-rational shift towards simpler, lower-cost strategies under time pressure. These results shed light on how people adapt their exploration and decision-making strategies to externally imposed cognitive constraints.

Asymmetric reinforcement learning facilitates human inference of transitive relations.

Humans and other animals are capable of inferring never-experienced relations (for example, A > C) from other relational observations (for example, A > B and B > C). The processes behind such transitive inference are subject to intense research. Here we demonstrate a new aspect of relational learning, building on previous evidence that transitive inference can be accomplished through simple reinforcement learning mechanisms. We show in simulations that inference of novel relations benefits from an asymmetric learning policy, where observers update only their belief about the winner (or loser) in a pair. Across four experiments (n = 145), we find substantial empirical support for such asymmetries in inferential learning. The learning policy favoured by our simulations and experiments gives rise to a compression of values that is routinely observed in psychophysics and behavioural economics. In other words, a seemingly biased learning strategy that yields well-known cognitive distortions can be beneficial for transitive inferential judgements.

Development of directed and random exploration in children.

Are young children just random explorers who learn serendipitously? Or are even young children guided by uncertainty-directed sampling, seeking to explore in a systematic fashion? We study how children between the ages of 4 and 9 search in an explore-exploit task with spatially correlated rewards, where exhaustive exploration is infeasible and not all options can be experienced. By combining behavioral data with a computational model that decomposes search into similarity-based generalization, uncertainty-directed exploration, and random exploration, we map out developmental trajectories of generalization and exploration. The behavioral data show strong developmental differences in children's capability to exploit environmental structure, with performance and adaptiveness of sampling decisions increasing with age. Through model-based analyses, we disentangle different forms of exploration, finding signature of both uncertainty-directed and random exploration. The amount of random exploration strongly decreases as children get older, supporting the notion of a developmental "cooling off" process that modulates the randomness in sampling. However, even at the youngest age range, children do not solely rely on random exploration. Even as random exploration begins to taper off, children are actively seeking out options with high uncertainty in a goal-directed fashion, and using inductive inferences to generalize their experience to novel options. Our findings provide critical insights into the behavioral and computational principles underlying the developmental trajectory of learning and exploration.

Integration and segregation across large-scale intrinsic brain networks as a marker of sustained attention and task-unrelated thought.

Sustained attention is a fundamental cognitive process that can be decoupled from distinct external events, and instead emerges from ongoing intrinsic large-scale network interdependencies fluctuating over seconds to minutes. Lapses of sustained attention are commonly associated with the subjective experience of mind wandering and task-unrelated thoughts. Little is known about how fluctuations in information processing underpin sustained attention, nor how mind wandering undermines this information processing. To overcome this, we used fMRI to investigate brain activity during subjects' performance (n=29) of a cognitive task that was optimized to detect and isolate continuous fluctuations in both sustained attention (via motor responses) and task-unrelated thought (via subjective reports). We then investigated sustained attention with respect to global attributes of communication throughout the functional architecture, i.e., by the segregation and integration of information processing across large scale-networks. Further, we determined how task-unrelated thoughts related to these global information processing markers of sustained attention. The results show that optimal states of sustained attention favor both enhanced segregation and reduced integration of information processing in several task-related large-scale cortical systems with concurrent reduced segregation and enhanced integration in the auditory and sensorimotor systems. Higher degree of mind wandering was associated with losses of the favored segregation and integration of specific subsystems in our sustained attention model. Taken together, we demonstrate that intrinsic ongoing neural fluctuations are characterized by two converging communication modes throughout the global functional architecture, which give rise to optimal and suboptimal attention states. We discuss how these results might potentially serve as neural markers for clinically abnormal attention. SIGNIFICANCE STATEMENT: Most of our brain activity unfolds in an intrinsic manner, i.e., is unrelated to immediate external stimuli or tasks. Here we use a gradual continuous performance task to map this intrinsic brain activity to both fluctuations of sustained attention and mind wandering. We show that optimal sustained attention is associated with concurrent segregation and integration of information processing within many large-scale brain networks, while task-unrelated thought is related to sub-optimal information processing in specific subsystems of this sustained attention network model. These findings provide a novel information processing framework for investigating the neural basis of sustained attention, by mapping attentional fluctuations to genuinely global features of intra-brain communication.

Correction: Similarities and differences in spatial and non-spatial cognitive maps.

[This corrects the article DOI: 10.1371/journal.pcbi.1008149.].

Similarities and differences in spatial and non-spatial cognitive maps.

Learning and generalization in spatial domains is often thought to rely on a "cognitive map", representing relationships between spatial locations. Recent research suggests that this same neural machinery is also recruited for reasoning about more abstract, conceptual forms of knowledge. Yet, to what extent do spatial and conceptual reasoning share common computational principles, and what are the implications for behavior? Using a within-subject design we studied how participants used spatial or conceptual distances to generalize and search for correlated rewards in successive multi-armed bandit tasks. Participant behavior indicated sensitivity to both spatial and conceptual distance, and was best captured using a Bayesian model of generalization that formalized distance-dependent generalization and uncertainty-guided exploration as a Gaussian Process regression with a radial basis function kernel. The same Gaussian Process model best captured human search decisions and judgments in both domains, and could simulate realistic learning curves, where we found equivalent levels of generalization in spatial and conceptual tasks. At the same time, we also find characteristic differences between domains. Relative to the spatial domain, participants showed reduced levels of uncertainty-directed exploration and increased levels of random exploration in the conceptual domain. Participants also displayed a one-directional transfer effect, where experience in the spatial task boosted performance in the conceptual task, but not vice versa. While confidence judgments indicated that participants were sensitive to the uncertainty of their knowledge in both tasks, they did not or could not leverage their estimates of uncertainty to guide exploration in the conceptual task. These results support the notion that value-guided learning and generalization recruit cognitive-map dependent computational mechanisms in spatial and conceptual domains. Yet both behavioral and model-based analyses suggest domain specific differences in how these representations map onto actions.

What Are We Curious about?

What are we curious about? Dubey and Griffiths propose a rational theory of curiosity that unifies previously contradictory novelty-based and complexity accounts. It also paves the way for future investigations, such as studying approximate models of curiosity as well as what causes abnormal levels of exploration.

Searching for Rewards Like a Child Means Less Generalization and More Directed Exploration.

How do children and adults differ in their search for rewards? We considered three different hypotheses that attribute developmental differences to (a) children's increased random sampling, (b) more directed exploration toward uncertain options, or (c) narrower generalization. Using a search task in which noisy rewards were spatially correlated on a grid, we compared the ability of 55 younger children (ages 7 and 8 years), 55 older children (ages 9-11 years), and 50 adults (ages 19-55 years) to successfully generalize about unobserved outcomes and balance the exploration-exploitation dilemma. Our results show that children explore more eagerly than adults but obtain lower rewards. We built a predictive model of search to disentangle the unique contributions of the three hypotheses of developmental differences and found robust and recoverable parameter estimates indicating that children generalize less and rely on directed exploration more than adults. We did not, however, find reliable differences in terms of random sampling.

Make-or-Break: Chasing Risky Goals or Settling for Safe Rewards?

Humans regularly pursue activities characterized by dramatic success or failure outcomes where, critically, the chances of success depend on the time invested working toward it. How should people allocate time between such make-or-break challenges and safe alternatives, where rewards are more predictable (e.g., linear) functions of performance? We present a formal framework for studying time allocation between these two types of activities, and we explore optimal behavior in both one-shot and dynamic versions of the problem. In the one-shot version, we illustrate striking discontinuities in the optimal time allocation policy as we gradually change the parameters of the decision-making problem. In the dynamic version, we formulate the optimal strategy-defined by a giving-up threshold-which adaptively dictates when people should stop pursuing the make-or-break goal. We then show that this strategy is computationally inaccessible for humans, and we explore boundedly rational alternatives. We compare the performance of the optimal model against (a) a myopic giving-up threshold that is easier to compute, and even simpler heuristic strategies that either (b) only decide whether or not to start pursuing the goal and never give up or (c) consider giving up at a fixed number of control points. Comparing strategies across environments, we investigate the cost and behavioral implications of sidestepping the computational burden of full rationality.

Generalization and Search in Risky Environments.

How do people pursue rewards in risky environments, where some outcomes should be avoided at all costs? We investigate how participant search for spatially correlated rewards in scenarios where one must avoid sampling rewards below a given threshold. This requires not only the balancing of exploration and exploitation, but also reasoning about how to avoid potentially risky areas of the search space. Within risky versions of the spatially correlated multi-armed bandit task, we show that participants' behavior is aligned well with a Gaussian process function learning algorithm, which chooses points based on a safe optimization routine. Moreover, using leave-one-block-out cross-validation, we find that participants adapt their sampling behavior to the riskiness of the task, although the underlying function learning mechanism remains relatively unchanged. These results show that participants can adapt their search behavior to the adversity of the environment and enrich our understanding of adaptive behavior in the face of risk and uncertainty.

Generalization guides human exploration in vast decision spaces.

From foraging for food to learning complex games, many aspects of human behaviour can be framed as a search problem with a vast space of possible actions. Under finite search horizons, optimal solutions are generally unobtainable. Yet, how do humans navigate vast problem spaces, which require intelligent exploration of unobserved actions? Using various bandit tasks with up to 121 arms, we study how humans search for rewards under limited search horizons, in which the spatial correlation of rewards (in both generated and natural environments) provides traction for generalization. Across various different probabilistic and heuristic models, we find evidence that Gaussian process function learning-combined with an optimistic upper confidence bound sampling strategy-provides a robust account of how people use generalization to guide search. Our modelling results and parameter estimates are recoverable and can be used to simulate human-like performance, providing insights about human behaviour in complex environments.

Asking better questions: How presentation formats influence information search.

While the influence of presentation formats have been widely studied in Bayesian reasoning tasks, we present the first systematic investigation of how presentation formats influence information search decisions. Four experiments were conducted across different probabilistic environments, where subjects (N = 2,858) chose between 2 possible search queries, each with binary probabilistic outcomes, with the goal of maximizing classification accuracy. We studied 14 different numerical and visual formats for presenting information about the search environment, constructed across 6 design features that have been prominently related to improvements in Bayesian reasoning accuracy (natural frequencies, posteriors, complement, spatial extent, countability, and part-to-whole information). The posterior variants of the icon array and bar graph formats led to the highest proportion of correct responses, and were substantially better than the standard probability format. Results suggest that presenting information in terms of posterior probabilities and visualizing natural frequencies using spatial extent (a perceptual feature) were especially helpful in guiding search decisions, although environments with a mixture of probabilistic and certain outcomes were challenging across all formats. Subjects who made more accurate probability judgments did not perform better on the search task, suggesting that simple decision heuristics may be used to make search decisions without explicitly applying Bayesian inference to compute probabilities. We propose a new take-the-difference (TTD) heuristic that identifies the accuracy-maximizing query without explicit computation of posterior probabilities. (PsycINFO Database Record